fix(coding-agent): use embedded Bun for xz-dev standalone packages - #5

@JohnsonRan — additional verification results and required acceptance before merge.

Please keep this PR unmerged until a standalone bundle built from the final corrected head passes actual installation, loading, and functional smoke tests for BOTH pi-notify and pi-subagents. Unit tests, mocked package-manager tests, --help, --list-models, tool discovery, and an installer returning 0 are not substitutes. Please diagnose the failures, implement the fixes, and continue testing until the required behavior works; do not hand the technical troubleshooting back to the maintainer.

1. What was verified on this head

Verified Pi head: 627c95875c774dd3e0c2a1a04ac4c51e51491de9, unchanged throughout the campaign.

Environment: rootless Podman, Debian bookworm, Git 2.39.5, public pi on PATH, embedded Bun 1.4.0, no external node, nodejs, npm, npx, bun, pnpm, or yarn on the execution PATH, fresh home/config, no host credentials or paid provider calls.

Tested bundle version: 0.85.1-xz.pr5.g627c9587.

• Native executable SHA-256: c5b693329d94c6f630556e19bd9f3cfb207e3e9b1072d5128670ad5cc16d9e73
• Bundle ZIP SHA-256: 28005c61c9f97a778801c82071ae8118f591206831224393336500125eb6f00f
• Container image: docker.io/library/debian@sha256:5ae3c39ebd15e229dcedd5cee596b2497182493d41ff162e824ba13fc1b2b867

These results belong to that head/bundle, not a future revision. Earlier missing-model-data/unbuilt-dist failures and the excluded Bun 1.3.14 build were retained separately. They were not fixed by editing tracked source. The full cross-platform suite was not run; this comment does not waive other required checks.

2. Blocking real-plugin failure

Required reproduction plugin:

• Repository: https://github.com/xz-dev/pi-notify
• Pin: 2d69229acd3037c31d4e9d566d02f6c0b92f24b9
• Source: git:github.com/xz-dev/pi-notify@2d69229acd3037c31d4e9d566d02f6c0b92f24b9
• Declared Git runtime dependency: pi-extension-utils at fc15bcfa8bc2f5ad56fe5db69137c9a0e29fb6b0.

Observed sequence:

pi install git:github.com/xz-dev/pi-notify@2d69229acd3037c31d4e9d566d02f6c0b92f24b9
# exit 0; exact checkout/source registered

bun install v1.4.0
+ typebox@1.3.11
+ @earendil-works/pi-coding-agent@0.84.1
137 packages installed
Blocked 2 postinstalls.

# Actual offline RPC startup, loading the installed package:
Error: Failed to load extension ".../pi-notify/index.ts": Failed to load extension: Cannot find module 'pi-extension-utils/semantic-hook'
Require stack:
- .../pi-notify/index.ts
# exit 1

The original package.json, original package-lock.json, and migrated bun.lock all contain the Git dependency. It is absent from the installed module tree. A stale/missing input-lock declaration has not been demonstrated. Also, the installed Pi package is declared as both a peer and dev dependency, so its presence alone does not prove dev-dependency leakage.

Please trace the actual Bun/install/migration behavior and fix the owning layer. Relevant integration points: getGitDependencyInstallArgs and installGit. Do not manufacture a pass by deleting lockfiles, manually installing the missing dependency, silently falling back to another manager, relaxing script trust, or substituting an easier plugin/version.

3. Linux metadata/cwd issue also needs resolution

The compiled Bun 1.4.0 command below fails without a cwd manifest and succeeds with one:

BUN_BE_BUN=1 pi info 'fixture-ext@^1.0.0' version --json

# No package.json:
error: Bun could not find a package.json file to install from
# exit 1

# With package.json:
"1.1.0"
# exit 0

This is not only the Windows limitation described in the PR body:

• getLatestNpmVersion runs in this.cwd, which may have no manifest.
• npmHasAvailableUpdate catches that failure and returns false.
• shouldUpdateNpmSource catches it and proceeds with installation, bypassing the successful-lookup version comparison.

Make metadata lookup work without requiring the user's project to be a Node project, while preserving registry/configuration and cwd semantics. Add a real standalone case with an installed version greater than the registry target and prove no downgrade. An actual downgrade was not reproduced here; exact-pin retention and ordinary range selection do not prove that adversarial case.

4. Required real plugins and functional tests

A. pi-notify

Keep the failing pin above as the regression input. Required evidence:

1. Install through normal public pi install in a clean no-external-manager environment.
2. Verify exact Git revision, preserved configured source, installed runtime dependency versions/resolution, and registration.
3. Load through normal Pi startup using the installed tree. Require RPC startup success and an actual get_state response, not merely no stderr or --list-models exit 0.
4. In a separate functional case, enable exactly one safe test-local notification action, exercise the real agent_notify/semantic-hook path, and assert its observable result, such as a marker in the isolated test directory. Supply the exact config and triggering harness. Do not send desktop/webhook notifications or load the operator's configuration.
5. Update the pinned installed source, restart/load it again, and verify it remains usable. Removal must remove its registration without affecting another installed package.

The diagnostic load recipe below uses empty actions intentionally. That proves loading, not notification functionality; item 4 is additionally required.

B. xz-dev/pi-subagents — this exact repository, not another project

The required repository is exactly https://github.com/xz-dev/pi-subagents. Do not substitute nicobailon/pi-subagents, another fork, or an unrelated npm package/project with the same or a similar name. Record the installed checkout's git remote get-url origin and full git rev-parse HEAD so the repository identity is independently verifiable. This is additional required acceptance, not a result claimed passed by the completed campaign.

• Repository: https://github.com/xz-dev/pi-subagents
• Pinned input for this request: 88639462aae9ba97465b203a305f3abf66bd195c (0.66.0, current main when this comment was prepared).
• Source: git:github.com/xz-dev/pi-subagents@88639462aae9ba97465b203a305f3abf66bd195c.

Required behavior is actual native subagent execution, not registration or a successful subagent({action:"list"}):

1. Install this exact GitHub source with the candidate's public pi install; require origin to identify xz-dev/pi-subagents, verify the full pinned checkout SHA, dependency resolution, registration, and extension loading in its own fresh home.
2. Launch a real bounded read-only native background child through the public subagent tool. Have it read a uniquely generated fixture value and return that value. Use a deterministic local/faux provider so no paid provider or operator credentials are required; the public tool, child process, embedded SDK/session factory, and runner must be real, not mocked.
3. Keep the parent alive. Verify real child completion, result content, status/artifacts, and delivery through the normal parent notification boundary. A saved result file alone does not prove completion delivery or process exit.
4. Verify cancellation/timeout and resume/revival boundaries, with observed child process exit and no lingering children or leaked state. Reuse the extension's existing lifecycle tests rather than inventing a substitute runner.
5. Verify the installed extension still works after the package update/reload path. Also run a combined clean-home smoke with both real extensions installed, so isolation-only success does not conceal a conflict.

The pinned extension already documents a deterministic 18-mode real standalone lifecycle matrix: standalone-background.md, with entrypoints test/smoke/standalone-matrix.mjs and test/smoke/standalone-background.mjs. Use this existing coverage for the lifecycle gate.

Important: that harness currently pins an official Pi release. Passing it with the stock official executable does not validate this PR. Provide an isolated candidate-bound invocation/input manifest for the exact newly built bundle, retaining hash checks, sandbox checks, real execution/positive controls, and observed-exit checks. Do not weaken those checks or relabel an official-binary receipt. If the matrix stages a source package, report that separately and also run the public-install → installed-tree → real-child smoke above.

A local/faux provider is acceptable; a fake child result, fake SDK, fake subagent tool, Node-backed fake pi, or hidden external runtime in the standalone execution environment is not.

If an extension itself genuinely needs a coordinated fix, identify the owner/root cause, link the fix, and report old/new pins explicitly. Do not silently switch to another revision to avoid the failing case.

set -euo pipefail
REPO="$PWD"
HEAD="$(git rev-parse HEAD)"
RUN="$(mktemp -d /tmp/pi-pr5-smoke.XXXXXX)"
mkdir -p "$RUN/logs" "$RUN/bun"
: > "$RUN/npmrc"

# Retain exact argv, both output streams, and the real exit status.
run_logged() {
  local name="$1" rc=0
  shift
  printf '%q ' "$@" > "$RUN/logs/$name.command"
  printf '\n' >> "$RUN/logs/$name.command"
  "$@" > "$RUN/logs/$name.stdout" 2> "$RUN/logs/$name.stderr" || rc=$?
  printf '%s\n' "$rc" > "$RUN/logs/$name.exit"
  return "$rc"
}

git diff --exit-code
git diff --cached --exit-code
git ls-files -z | sort -z | xargs -0 sha256sum > "$RUN/source-before.sha256"
printf '%s\n' "$HEAD" > "$RUN/head.txt"

# Current candidate requires Bun 1.4.0. Do not accidentally use host Bun 1.3.x.
curl -fL https://github.com/oven-sh/bun/releases/download/bun-v1.4.0/bun-linux-x64.zip -o "$RUN/bun/bun-linux-x64.zip"
curl -fL https://github.com/oven-sh/bun/releases/download/bun-v1.4.0/SHASUMS256.txt -o "$RUN/bun/SHASUMS256.txt"
(cd "$RUN/bun"; awk '$2 == "bun-linux-x64.zip"' SHASUMS256.txt | sha256sum --check -)
unzip -q "$RUN/bun/bun-linux-x64.zip" -d "$RUN/bun"
BUN_BIN="$RUN/bun/bun-linux-x64"
test "$("$BUN_BIN/bun" --version)" = 1.4.0
run_logged deps env NPM_CONFIG_USERCONFIG="$RUN/npmrc" npm ci --ignore-scripts

# On this revision this invokes generate-models.ts --strict --data-only.
# Only ignored JSON may change; no tracked model-code repair.
run_logged hydrate env NPM_CONFIG_USERCONFIG="$RUN/npmrc" npm run hydrate:model-data
VERSION="$(node -p 'require("./packages/coding-agent/package.json").version')"
run_logged build env PATH="$BUN_BIN:$PATH" NPM_CONFIG_USERCONFIG="$RUN/npmrc" \
  bash scripts/build-binaries.sh --skip-install --offline-model-data \
  --platform linux-x64-gnu-modern --out "$RUN/release" \
  --distribution-version "${VERSION}-xz.pr5.g${HEAD:0:8}"
run_logged check env NPM_CONFIG_USERCONFIG="$RUN/npmrc" npm run check
(cd "$REPO/packages/coding-agent"; run_logged focused \
  node "$REPO/node_modules/vitest/dist/cli.js" --run \
  test/package-manager.test.ts test/package-manager-ssh.test.ts test/package-command-paths.test.ts)
git diff --check
git ls-files -z | sort -z | xargs -0 sha256sum > "$RUN/source-after.sha256"
cmp "$RUN/source-before.sha256" "$RUN/source-after.sha256"

BUNDLE="$RUN/release/linux-x64-gnu-modern"
sha256sum "$BUNDLE/pi" "$BUNDLE/pi-native" "$RUN/release/pi-linux-x64-gnu-modern.zip" > "$RUN/bundle.sha256"
IMAGE="docker.io/library/debian@sha256:5ae3c39ebd15e229dcedd5cee596b2497182493d41ff162e824ba13fc1b2b867"
NAME="pi-pr5-smoke-${HEAD:0:8}-$$"
run_logged container-create podman run -d --name "$NAME" \
  --security-opt=no-new-privileges "$IMAGE" sleep infinity
run_logged container-setup podman exec "$NAME" /bin/sh -ec \
  'apt-get update; DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git ca-certificates; mkdir -p /opt/pi /work/project /work/home/.pi/agent'
run_logged bundle-copy podman cp "$BUNDLE/." "$NAME:/opt/pi"

CENV=(env -i HOME=/work/home PI_CODING_AGENT_DIR=/work/home/.pi/agent \
  PATH=/opt/pi:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
run_logged environment podman exec -w /work/project "$NAME" "${CENV[@]}" /bin/sh -ec '
  command -v pi
  pi --version
  BUN_BE_BUN=1 pi --version
  git --version
  for tool in node nodejs npm npx bun pnpm yarn; do
    if command -v "$tool"; then echo "Unexpected external manager: $tool" >&2; exit 1; fi
    printf "%s=ABSENT\n" "$tool"
  done
'

PIN=2d69229acd3037c31d4e9d566d02f6c0b92f24b9
SOURCE="git:github.com/xz-dev/pi-notify@$PIN"
EXT=/work/home/.pi/agent/git/github.com/xz-dev/pi-notify
printf '%s\n' '{"events":{"agent_settled":{"actions":[]},"tool_execution_start:ask_user_question":{"actions":[]}},"hooks":{"agent-notify":{"actions":[]},"user-ready":{"actions":[]}}}' > "$RUN/pi-notify.json"
run_logged inert-config podman cp "$RUN/pi-notify.json" "$NAME:/work/home/.pi/agent/pi-notify.json"

# Keep failed stages as failures; still collect the independent load evidence.
run_logged install podman exec -w /work/project "$NAME" "${CENV[@]}" pi install "$SOURCE" || :
run_logged list podman exec -w /work/project "$NAME" "${CENV[@]}" pi list || :
run_logged revision podman exec "$NAME" git -C "$EXT" rev-parse HEAD || :
run_logged dependency podman exec -w "$EXT" "$NAME" "${CENV[@]}" BUN_BE_BUN=1 pi -e \
  'import("pi-extension-utils/semantic-hook").then(()=>console.log("dependency-import-ok")).catch(e=>{console.error(e);process.exit(1)})' || :

# Normal autoload from installed registration; no --no-extensions workaround.
printf '%s\n' '{"id":"smoke","type":"get_state"}' > "$RUN/request.jsonl"
run_logged rpc timeout 30s podman exec -i -w /work/project "$NAME" \
  "${CENV[@]}" PI_OFFLINE=1 pi --mode rpc --no-session --no-skills \
  --no-prompt-templates --no-themes --offline < "$RUN/request.jsonl" || :
run_logged settings-copy podman cp "$NAME:/work/home/.pi/agent/settings.json" "$RUN/settings.json" || :

# Host-side assertions; Node is not installed in the execution container.
run_logged load-verdict node --input-type=module - "$RUN" "$PIN" "$SOURCE" <<'JS' || :
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const [root, pin, source] = process.argv.slice(2);
const read = (name) => readFileSync(`${root}/logs/${name}`, "utf8");
for (const stage of ["install", "list", "revision", "dependency", "rpc", "settings-copy"]) {
  assert.equal(read(`${stage}.exit`).trim(), "0", `${stage} failed; inspect its logs`);
}
assert.equal(read("revision.stdout").trim(), pin);
assert.match(read("dependency.stdout"), /dependency-import-ok/);
const settings = JSON.parse(readFileSync(`${root}/settings.json`, "utf8"));
assert.deepEqual(settings.packages, [source]);
const replies = read("rpc.stdout").split("\n").filter(Boolean).map((line) => JSON.parse(line));
assert(replies.some((r) => r.id === "smoke" && r.type === "response" &&
  r.command === "get_state" && r.success === true && r.data.messageCount === 0));
assert.doesNotMatch(read("rpc.stderr"), /Failed to load extension|Cannot find module/);
console.log("real-plugin-load-ok");
JS

printf 'Retained evidence: %s\nContainer: %s\n' "$RUN" "$NAME"
printf 'Load verdict exit: '; cat "$RUN/logs/load-verdict.exit"
exit "$(cat "$RUN/logs/load-verdict.exit")"

5. Required reproducible handoff with the fix

Please attach or link artifacts accessible to reviewers, not only local /tmp paths:

• Identity: final Pi commit; both plugin repository URLs, full commit pins and versions; relevant Git dependency revisions; OS/architecture; container image digest; Node/npm used for provisioning; Bun pin; public Pi version; bundle/executable SHA-256.
• Runnable scripts: exact build/install/load/functional commands; fixture source; agent definition/configuration; deterministic provider setup; notification action and trigger; registry/Git server configuration; deadlines and cleanup. No undisclosed manual repair step.
• Complete evidence: stdout, stderr, exact argv/cwd, exit status for every stage, selected sanitized environment, dependency/lock manifests, configured sources, actual extension load evidence, child output/status/notification/observed-exit evidence, and final resource inventory.
• Acceptance table: each real plugin's install → dependencies → registration → actual load → actual functionality → update/reload → removal result; combined-use result; explicit PASS/FAIL/NOT RUN. Keep expected negative controls separate from failed positive requirements.
• Regression coverage: the three focused Pi suites plus full check on the same final head; manifest-less metadata lookup; genuine no-downgrade case; scoped/range/tag/pin behavior; changed/unchanged Git revisions, failed installation cleanup and retry; public PATH and explicit override boundaries. Controlled registry metadata must match its tarball dependencies.
• Failure history: retain the original red result and each changed-fixture rerun. Do not combine stages from different candidate/package snapshots into one green receipt. Document native/build-script limits honestly; recording a limitation is not a successful functional test.

6. Ownership and scope

Please own the technical investigation end to end: read the Pi/Bun/extension source and existing tests, choose the appropriate runner and test seams, find the root cause, implement the smallest fix, and keep rerunning the failing checks until both real plugins meet the acceptance criteria. The maintainer should not have to choose package-manager flags, debug fixture URLs, design the SDK bootstrap, or supply implementation details. If local infrastructure is unsuitable, use an appropriate existing runner within your access instead of returning the technical problem to the user.

Only escalate a genuine authorization/credential/cost/privilege or scope decision. State the exact blocker and minimal permission required; do not bypass authentication or safety boundaries. A technical failure by itself is a task to investigate, not a reason to stop and ask a non-specialist what implementation to try.

Keep the separate runtime-distribution-gate removal out of this patch. Persistent patch/* delivery must use a clean upstream-based feature history; changing this PR's base alone will not remove its merged downstream-main ancestry. Agree the delivery operation separately rather than silently retargeting or rewriting it.

This comment is AI-generated by Pi at the maintainer's request.

Follow-up to the verification above: pushed d0c7f841 and 68730f34. This update adds only the Bun patch, regressions, and local-runtime metadata under patches/. CI/workflow changes remain local.

• Local verification used Bun 1.4.2, which includes the Git-dependency installation fix, plus the projectless info/pm view patch. Original plugin manifests/npm locks stayed unchanged; no cwd/config-copy workaround or generated manifest.
• The frozen Linux candidate passed actual pi-notify/pi-subagents installation and loading, a notification action, native child fixture read, normal completion delivery, and observed exit. The existing 18-mode matrix, plugin lifecycle checks, and no-downgrade checks passed.
• npm run check, 23 Bun metadata tests, and 215 focused package-manager tests passed. Candidate identity and reproduction limits are in patches/README.md; artifacts/receipts remain local.

Not merge/release-ready: the locally built runtime requires GLIBC 2.43, versus official Bun's 2.17 baseline. A compatible runtime/final candidate still needs acceptance, and release integration is not included here. Full Windows suite and remaining platform/release gates remain outstanding. Local functional results do not close those blockers.

This comment is AI-generated.

Updated to eb918d4e0, merging current main without rewriting history. The PR diff is now 7 files.

Exact-head Linux acceptance passed with official Bun 1.4.2, including 230 focused tests and real package install/update/repair/removal, installed-plugin actions, and native child completion/observed exit without external Node/npm/Bun.

Projectless metadata queries still fail, and release workflows still pin Bun 1.4.0. This is core package-management acceptance, not release readiness. The updated PR description contains artifact hashes, evidence, and remaining limitations.

This comment is AI-generated.

Implemented and released through the independently maintainable downstream patch, preserving the core implementation from this PR:

• Runtime: patch/use-embedded-bun-package-manager, commits d1674ce70d0d63f8f174f08499b2adfd044d4c5b and 6b164ce637ea1b0d11e3b8edd4b805a48aae576b, based on upstream 400d6905ce46ec46e79da8a7701b1b48850192df.
• CI integration: a58780d37910a9ade8badd7eb7a310917a1043a3 and f4d852e7f3b243668682a0e8cf26e590e258256e on ci.
• Generated main: 53f2fd83acc1348582c1c6313ef43213182ad8d9.
• Release 147; successful release workflow.

Two fixes were required beyond the initial implementation:

1. Use official Bun 1.4.2 for the npm-lock Git-dependency installation fix. Projectless metadata remains unsupported. With the owner's approval, embedded-default updates now stop with an error when an installed package's target version cannot be verified, rather than risking a downgrade. Explicit manager overrides retain their prior policy. No custom Bun runtime or --no-save workaround is shipped.
2. Use bun update for embedded managed npm updates; bun install retained an older locked version within a configured range. Release 146 exposed this during final verification and is superseded by 147.

The downloaded public Linux x64 GNU archive (SHA-256 f310cd3ca13e27b74f08f1ff7a2daeb662e2ccd01320149e1c0b302ec9a13eed) passed provenance verification and installed-tree checks for both exact plugin pins, real pi-notify action delivery, native child fixture read/result delivery/observed exit, update/reload/removal, cold-cache no-downgrade, ranged/tagged/pinned npm updates, Git update/repair/failure retry, PATH wrappers and script-trust boundaries. The repeatable primary gate is node scripts/e2e-embedded-package-manager.mjs <public-pi> <fresh-receipt-dir> and runs against the final Linux archive before publication.

All 12 platform build/archive/self-update jobs passed. The 18 lifecycle scenario categories were additionally exercised on Linux using an installed-tree adaptation of the pinned subagents tests, not the unmodified official-binary-hash/bubblewrap driver; do not interpret this as identical upstream harness certification or cross-platform plugin coverage. Native packages needing unavailable build tooling remain a documented compatibility limit.

Closing as superseded by the shipped clean patch rather than merging the PR's inherited generated-main and unrelated overlay history. The runtime and packaging changes remain on their respective persistent branches.

This comment is AI-generated by the Pi coding assistant.

Final verification update: Release 147 now also passed all 18 lifecycle modes using the unmodified test/smoke/standalone-background.mjs from pinned xz-dev/pi-subagents 88639462aae9ba97465b203a305f3abf66bd195c. Every mode ran with bubblewrap network/PID isolation, a failing bare-runtime SDK import negative control, no staged/downloaded core SDK/shim, empty Bun cache, and real shutdown/exit checks. Only the outer matrix identity binding was changed to the downloaded fork Release native SHA-256 5d600100e60542e5792f449c1107a7da54583404cd2302bb538dc5ccbdaf8cc6. All modes used package SHA-256 53180058220192402d736e4d688b8d372dc97cb0aeeabab4a780b3017f1cf1db.

OpenSpec is synchronized and archived on plan/use-embedded-bun-package-manager at 5d1ff680b. The verification report records the final Release, requirement coverage, corrected false leads, compatibility limits, and the separately authorized docs-only pre-commit exception on the old planning baseline.

Archived verification report

This comment is AI-generated by the Pi coding assistant.